Skip to content

Stream the agent loop end to end and add markdown skills#6

Merged
folathecoder merged 3 commits into
mainfrom
agentling/feature/skills
Jul 7, 2026
Merged

Stream the agent loop end to end and add markdown skills#6
folathecoder merged 3 commits into
mainfrom
agentling/feature/skills

Conversation

@folathecoder

Copy link
Copy Markdown
Owner

Day 6: streaming plus a progressive-disclosure skill system.

Streaming

  • The loop now drives every model turn through model.stream and rebuilds the
    turn with agglomerate_deltas, emitting TextDelta as tokens arrive (both
    the main step and the step-limit forced answer).
  • print_events(): a small live CLI renderer for the event stream.

Skills

  • skills.py: a Skill is a folder with a SKILL.md (YAML frontmatter plus a
    markdown body). Skill.from_path parses and validates it; load_tools
    resolves declared module:attribute entry points to Tool objects.
  • Progressive disclosure: only skill names and descriptions go in the system
    prompt. The built-in load_skill(name) tool reveals a skill's body and
    registers its tools on demand, so context stays small until a skill is used.
  • A code-reviewer example skill under examples/skills/.

Review fixes folded in

  • Guard max_steps < 1 instead of silently using the default.
  • Reject duplicate tool names at construction.
  • run() overloads: stream=False types as Awaitable[str], stream=True as
    AsyncIterator[Event].

33 new tests (98 total); mypy and ruff clean.

Closes FOL-43.

A Skill is a folder holding a SKILL.md: YAML frontmatter (name, description,
and optional tool entry points) followed by a markdown instruction body.
Skill.from_path parses and validates it; load_tools resolves declared
"module:attribute" entry points to Tool objects. Adds pyyaml and a
code-reviewer example skill.
Drive each model turn through model.stream and agglomerate_deltas, emitting
TextDelta as tokens arrive. Register skills with progressive disclosure: only
names and descriptions go in the system prompt, and the built-in load_skill
tool reveals a skill's body and registers its tools on demand.

Also fold in three review fixes:
- Guard max_steps < 1 instead of silently falling back to the default.
- Reject duplicate tool names at construction.
- Add run() overloads: stream=False types as Awaitable[str], stream=True as
  AsyncIterator[Event].
@folathecoder folathecoder merged commit a412a4d into main Jul 7, 2026
2 checks passed
@folathecoder folathecoder deleted the agentling/feature/skills branch July 7, 2026 10:24
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Stream agent loop end-to-end and add progressive markdown skills

✨ Enhancement 🧪 Tests ⚙️ Configuration changes 📝 Documentation 🕐 40+ Minutes

Grey Divider

AI Description

• Stream every model turn via model.stream, emitting TextDelta tokens as they arrive.
• Add progressively-disclosed markdown skills loaded on demand via a built-in load_skill tool.
• Improve construction safety (max_steps guard, duplicate tool rejection) and expand test coverage.
Diagram

graph TD
  A["Agent loop"] --> M[["Model.stream"]] --> G["agglomerate_deltas"]
  A --> E["Event stream"] --> P["print_events CLI"]
  A --> L["load_skill tool"] --> S[("SKILL.md")] --> T["Tool registry"]

  subgraph Legend
    direction LR
    _cmp["Component"] ~~~ _ext[["External/API"]] ~~~ _file[("File")]
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Use Python entry points (importlib.metadata) for skill tools
  • ➕ Avoids stringly-typed "module:attribute" specs in markdown
  • ➕ Supports plugin discovery without hardcoding module paths
  • ➖ More packaging overhead for users authoring local skills
  • ➖ Harder to use for ad-hoc, repo-local skills during prototyping
2. Adopt a dedicated frontmatter parser library
  • ➕ More robust parsing across edge cases (BOMs, fence variants, etc.)
  • ➕ May reduce custom parsing/validation code
  • ➖ Adds another dependency and potential supply-chain surface
  • ➖ Current _split_frontmatter is small and already well-tested

Recommendation: The current approach is a good fit for a lightweight framework: YAML frontmatter + module:attribute keeps authoring simple and the progressive-disclosure load_skill tool directly addresses context pressure. Consider entry points only if you expect a plugin ecosystem beyond repo-local skills.

Files changed (7) +820 / -22

Enhancement (3) +313 / -18
agent.pyStream model turns and add progressive skill loading +159/-18

Stream model turns and add progressive skill loading

• Routes all model turns (including step-limit forced final answer) through 'model.stream', emitting 'TextDelta' events and rebuilding the final message via 'agglomerate_deltas'. Adds progressive skills: accepts 'Skill|str|Path', appends a catalog (name/description only) to the system prompt, and provides a built-in 'load_skill' tool that reveals full instructions and registers skill tools idempotently. Also adds construction guards for 'max_steps < 1', rejects duplicate tool names, and tightens 'run()' typing via overloads.

src/agentling/agent.py

events.pyAdd 'print_events()' live renderer for streamed events +45/-0

Add 'print_events()' live renderer for streamed events

• Adds a small CLI-oriented consumer for the agent event stream that prints streamed text incrementally and formats tool calls/results and the final answer. Includes output truncation to keep tool output readable during live streaming.

src/agentling/events.py

skills.pyIntroduce markdown Skill loader with YAML frontmatter +109/-0

Introduce markdown Skill loader with YAML frontmatter

• Adds 'Skill' dataclass and 'Skill.from_path()' to parse 'SKILL.md' (YAML frontmatter + markdown body) and validate required keys. Implements tool entry point resolution ('module:attribute') to 'Tool' instances and frontmatter splitting with clear error paths.

src/agentling/skills.py

Tests (2) +472 / -4
test_agent.pyExtend agent tests for streaming text deltas and skills +191/-4

Extend agent tests for streaming text deltas and skills

• Updates the FakeModel to support streaming and adds assertions for emitted 'TextDelta' chunks. Adds progressive skill tests covering catalog injection, load_skill behavior (body reveal, tool registration, unknown skill errors), path-based skill loading, and construction guards for duplicate tools and invalid max_steps.

tests/test_agent.py

test_skills.pyAdd comprehensive unit tests for Skill parsing and tool resolution +281/-0

Add comprehensive unit tests for Skill parsing and tool resolution

• Adds coverage for 'Skill.from_path', '_split_frontmatter', '_resolve_tool', and 'Skill.load_tools', including multiple error paths and an end-to-end test that loads the bundled 'code-reviewer' example. Ensures frontmatter parsing preserves body content like horizontal rules.

tests/test_skills.py

Documentation (1) +33 / -0
SKILL.mdAdd bundled code-reviewer example skill +33/-0

Add bundled code-reviewer example skill

• Introduces an example markdown skill with YAML frontmatter (name/description) and a structured instruction body for code review behavior. Serves as a reference skill folder layout under 'examples/skills/'.

examples/skills/code-reviewer/SKILL.md

Other (1) +2 / -0
pyproject.tomlAdd PyYAML runtime and typing dependencies +2/-0

Add PyYAML runtime and typing dependencies

• Adds 'pyyaml' as a project dependency to parse SKILL.md frontmatter and 'types-PyYAML' to the dev extras for type checking. Keeps the skill loader dependency explicit and mypy-friendly.

pyproject.toml

@qodo-code-review

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Skill tools type coercion 🐞 Bug ≡ Correctness
Description
Skill.from_path() coerces frontmatter.tools via list(...), so a YAML scalar string becomes a list of
characters and later load_tools() feeds invalid specs into _resolve_tool(), breaking skill loading.
name/description/tools also aren’t type-validated, so non-string YAML values can create
hard-to-debug runtime failures (e.g., skill lookup mismatches).
Code

src/agentling/skills.py[R50-64]

+        try:
+            name = frontmatter["name"]
+            description = frontmatter["description"]
+        except KeyError as exc:
+            raise ValueError(
+                f"{folder / SKILL_FILE} is missing required frontmatter key {exc}."
+            ) from exc
+
+        return cls(
+            name=name,
+            description=description,
+            instructions=body.strip(),
+            path=folder,
+            tools=list(frontmatter.get("tools") or []),
+        )
Evidence
The current parsing assigns raw YAML values into the Skill object and coerces tools with
list(...), which turns scalar strings into character lists. load_tools() then resolves each
element as a module:attribute entry point, so character specs fail deterministically.

src/agentling/skills.py[46-64]
src/agentling/skills.py[66-109]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`Skill.from_path()` does not validate the types of `name`, `description`, or `tools` from YAML frontmatter, and it normalizes `tools` with `list(frontmatter.get("tools") or [])`. If `tools` is provided as a YAML scalar (string), `list(str)` produces a list of characters; subsequent `load_tools()` calls `_resolve_tool()` with invalid one-character specs and the skill fails to load.

### Issue Context
Skills are intended to be user-authored markdown files. YAML commonly allows either scalars or sequences, so this failure mode is plausible and will surface as runtime errors when the model calls `load_skill()`.

### Fix Focus Areas
- src/agentling/skills.py[43-64]
- src/agentling/skills.py[66-109]

### Implementation notes
- Validate `name` and `description` are `str` (raise `ValueError` with file context if not).
- Normalize `tools` robustly:
 - `None`/missing -> `[]`
 - `str` -> `[tools_value]`
 - `list`/`tuple` -> ensure all entries are `str`
 - otherwise -> `ValueError`
- Consider adding tests for `tools: some.module:tool` (scalar) and for non-string `name/description` values.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Skills silently overwritten 🐞 Bug ≡ Correctness
Description
Agent.__init__ stores skills in a dict keyed by skill.name without checking duplicates, so later
skills silently overwrite earlier ones. This can cause the agent to load a different skill than the
caller intended when duplicate names exist.
Code

src/agentling/agent.py[R89-91]

+        self.skills: dict[str, Skill] = {
+            skill.name: skill for skill in (_as_skill(entry) for entry in skills)
+        }
Evidence
The code constructs self.skills via a dict comprehension keyed by skill.name with no duplicate
detection; load_skill() then uses self.skills.get(name), so any overwrite directly changes what
gets loaded for the same name.

src/agentling/agent.py[86-96]
src/agentling/agent.py[307-333]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`Agent.__init__` builds `self.skills` with a dict comprehension keyed by `skill.name`, which silently overwrites earlier skills when duplicate names are provided. This is inconsistent with the new behavior for tools (which explicitly rejects duplicate tool names) and can lead to confusing, incorrect skill resolution.

### Issue Context
Skill names are part of the system prompt catalog and are used as the lookup key for the built-in `load_skill(name)` tool. Silent overwrites make debugging configuration errors difficult.

### Fix Focus Areas
- src/agentling/agent.py[86-96]

### Implementation notes
- Replace the dict comprehension with a loop that checks for duplicates and raises `ValueError(f"Duplicate skill name: {name!r}")`.
- Add/adjust tests to cover providing two skills with the same name.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Empty stream silent answer 🐞 Bug ☼ Reliability
Description
If Model.stream() yields zero deltas, _run_stream() passes an empty list to agglomerate_deltas(),
producing an assistant message with empty content and no tool calls; the agent then terminates with
an empty final answer. This fails silently instead of surfacing a model/transport failure.
Code

src/agentling/agent.py[R193-202]

+            deltas: list[Delta] = []
+            async for delta in self.model.stream(messages, tools=self._tool_schemas):
+                if delta.content:
+                    yield TextDelta(text=delta.content)
+                deltas.append(delta)
+
+            response = agglomerate_deltas(deltas)

            # Forgiving termination: no tool calls means the content is the answer.
            if not response.tool_calls:
Evidence
The agent’s streaming path always aggregates deltas and treats an aggregated message with no tool
calls as a final answer. The delta aggregation function returns empty content/tool_calls when given
an empty iterable, and the Model.stream contract does not state it must yield at least one Delta.

src/agentling/agent.py[188-206]
src/agentling/models.py[90-107]
src/agentling/models.py[375-417]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`Agent._run_stream()` assumes `Model.stream()` yields at least one delta. If it yields none, `agglomerate_deltas([])` returns a `ChatMessage` with `content=""` and `tool_calls=[]`, which the agent interprets as a valid terminal answer and returns an empty response.

### Issue Context
The `Model.stream()` protocol does not guarantee non-empty output. Empty output can occur due to provider bugs, early connection termination, or an adapter implementation mistake.

### Fix Focus Areas
- src/agentling/agent.py[188-206]
- src/agentling/agent.py[254-274]
- src/agentling/models.py[375-417]

### Implementation notes
- After the streaming loop, check `if not deltas:` and raise a clear exception (e.g., `RuntimeError("Model.stream produced no deltas")`) or otherwise surface an explicit failure.
- Apply the same guard to the step-limit forced-answer streaming block.
- Add a unit test with a Model.stream implementation that yields nothing and assert the agent raises (or emits a clear error outcome).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment thread src/agentling/skills.py
Comment on lines +50 to +64
try:
name = frontmatter["name"]
description = frontmatter["description"]
except KeyError as exc:
raise ValueError(
f"{folder / SKILL_FILE} is missing required frontmatter key {exc}."
) from exc

return cls(
name=name,
description=description,
instructions=body.strip(),
path=folder,
tools=list(frontmatter.get("tools") or []),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Skill tools type coercion 🐞 Bug ≡ Correctness

Skill.from_path() coerces frontmatter.tools via list(...), so a YAML scalar string becomes a list of
characters and later load_tools() feeds invalid specs into _resolve_tool(), breaking skill loading.
name/description/tools also aren’t type-validated, so non-string YAML values can create
hard-to-debug runtime failures (e.g., skill lookup mismatches).
Agent Prompt
### Issue description
`Skill.from_path()` does not validate the types of `name`, `description`, or `tools` from YAML frontmatter, and it normalizes `tools` with `list(frontmatter.get("tools") or [])`. If `tools` is provided as a YAML scalar (string), `list(str)` produces a list of characters; subsequent `load_tools()` calls `_resolve_tool()` with invalid one-character specs and the skill fails to load.

### Issue Context
Skills are intended to be user-authored markdown files. YAML commonly allows either scalars or sequences, so this failure mode is plausible and will surface as runtime errors when the model calls `load_skill()`.

### Fix Focus Areas
- src/agentling/skills.py[43-64]
- src/agentling/skills.py[66-109]

### Implementation notes
- Validate `name` and `description` are `str` (raise `ValueError` with file context if not).
- Normalize `tools` robustly:
  - `None`/missing -> `[]`
  - `str` -> `[tools_value]`
  - `list`/`tuple` -> ensure all entries are `str`
  - otherwise -> `ValueError`
- Consider adding tests for `tools: some.module:tool` (scalar) and for non-string `name/description` values.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment thread src/agentling/agent.py
Comment on lines +89 to +91
self.skills: dict[str, Skill] = {
skill.name: skill for skill in (_as_skill(entry) for entry in skills)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

2. Skills silently overwritten 🐞 Bug ≡ Correctness

Agent.__init__ stores skills in a dict keyed by skill.name without checking duplicates, so later
skills silently overwrite earlier ones. This can cause the agent to load a different skill than the
caller intended when duplicate names exist.
Agent Prompt
### Issue description
`Agent.__init__` builds `self.skills` with a dict comprehension keyed by `skill.name`, which silently overwrites earlier skills when duplicate names are provided. This is inconsistent with the new behavior for tools (which explicitly rejects duplicate tool names) and can lead to confusing, incorrect skill resolution.

### Issue Context
Skill names are part of the system prompt catalog and are used as the lookup key for the built-in `load_skill(name)` tool. Silent overwrites make debugging configuration errors difficult.

### Fix Focus Areas
- src/agentling/agent.py[86-96]

### Implementation notes
- Replace the dict comprehension with a loop that checks for duplicates and raises `ValueError(f"Duplicate skill name: {name!r}")`.
- Add/adjust tests to cover providing two skills with the same name.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment thread src/agentling/agent.py
Comment on lines +193 to 202
deltas: list[Delta] = []
async for delta in self.model.stream(messages, tools=self._tool_schemas):
if delta.content:
yield TextDelta(text=delta.content)
deltas.append(delta)

response = agglomerate_deltas(deltas)

# Forgiving termination: no tool calls means the content is the answer.
if not response.tool_calls:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

3. Empty stream silent answer 🐞 Bug ☼ Reliability

If Model.stream() yields zero deltas, _run_stream() passes an empty list to agglomerate_deltas(),
producing an assistant message with empty content and no tool calls; the agent then terminates with
an empty final answer. This fails silently instead of surfacing a model/transport failure.
Agent Prompt
### Issue description
`Agent._run_stream()` assumes `Model.stream()` yields at least one delta. If it yields none, `agglomerate_deltas([])` returns a `ChatMessage` with `content=""` and `tool_calls=[]`, which the agent interprets as a valid terminal answer and returns an empty response.

### Issue Context
The `Model.stream()` protocol does not guarantee non-empty output. Empty output can occur due to provider bugs, early connection termination, or an adapter implementation mistake.

### Fix Focus Areas
- src/agentling/agent.py[188-206]
- src/agentling/agent.py[254-274]
- src/agentling/models.py[375-417]

### Implementation notes
- After the streaming loop, check `if not deltas:` and raise a clear exception (e.g., `RuntimeError("Model.stream produced no deltas")`) or otherwise surface an explicit failure.
- Apply the same guard to the step-limit forced-answer streaming block.
- Add a unit test with a Model.stream implementation that yields nothing and assert the agent raises (or emits a clear error outcome).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant